A software Tool for Teaching Fault Tolerance in Artificial Neural Networks

This paper presents a software tool for teaching, evaluating and improving the fault tolerance of an Artificial Neural Network. Fault tolerance is a characteristic of parallel distributed systems such as neural networks. This property, though usually pointed out as one of the main advantages of the Neural Networks, is difficult to evaluate and can be a residual property allowing to maintain only a small part of the Neural Network characteristics. The fault tolerance of the Artificial Neural Networks (ANN) can be very important in critic applications such as the ones of nanotechnologies in interaction with the human body. In these situations, retaining a part of the characteristics of the Neural Network by fault tolerance or graceful degradation (GD) is crucial. To evaluate and teach the concepts of fault tolerance associated with neural networks (NNs) a software tool was developed. The tool is called Fault Tolerance Simulation and Evaluation Tool for Artificial Neural Networks (FTSET) and is composed of two main sub-tools (the Evaluator, for evaluation of the tolerance and the Improver, for improving the built-in tolerance). The FTSET is able to evaluate the fault tolerance in feedforward ANN of arbitrary size, though the bigger the network, the longer it takes to evaluate its characteristics. The tool can be used to evaluate fault tolerance from the set of weights of the network and the ranges of the inputs. The expression fault tolerance is in fact an abuse for most of the situations, since for real valued outputs what can be achieved is GD. Each fault does in fact affect the output, but for most of the situations the effect is not catastrophic and a concept of GD applies. The graceful degradation evaluation follows an exhaustive approach, testing all possible faults in the network (the faults mentioned here are physical faults that may happen in the hardware). It supplies a numerical analysis of the results obtained to the user and this information is then used by the Improver to build a NN with a higher GD degree. Changing the structure of an ANN to improve GD is a complex task since it may involve dealing with nonlinear functions such as the common activation functions hardlimit, hyperbolic tangent or other sigmoidal functions. (Dias & Antunes, 2007b) proposed a solution that involves duplicating part of the network (inputs, neurons, bias) to improve this characteristic, after the evaluation of the critical connections in terms of output impact. The Improver implements the ideas proposed in this paper. The proposed tool is very useful to teach the concepts of fault tolerance and GD and to illustrate these concepts associated with neural networks. The software tool presented here is the result of the final project of the Engineering degree (5 years) in Electronics and Computers and was implemented by a group of 2 students during a full year. The Evaluator sub-tool The faults are considered according to the models proposed in (Dias & Antunes, 2007a). The evaluation of the GD is done in percentage, according to the following expression: fault without output fault without output fault with output GD _ _ _ _ _ _ − = Eq. 1 To evaluate GD, the output of the network must be calculated in several different situations. Since the input can take several values and it is not possible to considerer a single value of the inputs to measure the effects of each fault, the limits in the range of values for each input will be used as a worst case value. After calculating the outputs for every possible combination of the inputs ranges, a matrix of possible values for the output is obtained. For each possible single fault, the network must be evaluated in the very same situations taken before simulating the fault, resulting for each fault in a new matrix that must be compared with the one obtained in the absence of faults. As can be easily observed this procedure has an exponential growth of calculations with the size of the network. In fact the number of calculations needed can be evaluated from the size of the network through the following expressions: The number of outputs to be calculated initially is: I X calc out n = _ _ Eq. 2 Where X is the number of possible values that the input can assume and I is the number of inputs. The total number of weights for a network of k layers is given by: O K K K K K K I weights n n n n * ) 1 ( * ) 1 ( ... * ) 1 ( * ) 1 ( _ 1 2 1 1 + + + + + + + + = − Eq. 3 Where Kj is the number of neurons in the layer j and O is the number of outputs. Equation 3 can be written as: O K K K K I weights n n n j j j * ) 1 ( * ) 1 ( * ) 1 ( _ 1 1 1 1 + + + + + = ∑ −